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Abstract 

Statistical mechanics approach is useful not only in analyzing macroscopic 
system performance of wireless communication systems, but also in dis- 
cussing design problems of wireless communication systems. In this paper, 
we discuss a design problem of spreading sequences in code-division multiple- 
access (CDMA) systems, as an example demonstrating the usefulness of sta- 
tistical mechanics approach. We analyze, via replica method, the average mu- 
tual information between inputs and outputs of a randomly- spread CDMA 
channel, and discuss the optimization problem with the average mutual in- 
formation as a measure of optimization. It has been shown that the average 
mutual information is maximized by orthogonally-invariant random Welch 
bound equality (WBE) spreading sequences. 

Key words: code-division multiple-access (CDMA), replica method, 
average mutual information, large system limit 



1. Introduction 

In recent years, advances in information and communication technologies 
have been demanding high data-rate wireless communications. In order to 
realize high data-rate wireless communications, bandwidth of systems should 
be as wide as possible, which means that such systems should have a large 
degree of freedom. Those systems are also required to be able to operate 
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efficiently even in bad and uncertain environments. For example, in urban 
areas, there are many obstacles, such as buildings, cars, and people, which 
interact with wireless communication systems as reflecting/scattering bod- 
ies, thereby making communication environment very complex. In analyzing 
wireless communication systems, therefore, one has to regard them as sys- 
tems with very high dimensionality and randomness. This is why statistical 
mechanics approach is expected to be useful in studying wireless communi- 
cation systems. 

In this paper, we consider a problem arising from considerations of multiple- 
access channels. Typically, a wireless communication system has to accom- 
modate multiple users simultaneously. In such a system, signals coming from 
different users interfere with each other. How to mitigate such multiple-access 
interference (MAI) is one of the most important problems in wireless commu- 
nications. Code-division multiple-access (CDMA) [i|,[2j provides an effective 
scheme to mitigate MAI, and is widely used in various commercial systems. 
In CDMA, an information symbol of a user is modulated with a spreading 
sequence assigned to the user. Receiver has to estimate information symbols 
based on received sequences by utilizing knowledge of spreading sequences of 
the users. 

A conventional choice to mitigate MAI is to use pseudorandom sequences 
as the spreading sequences. Although analysis of such randomly-spread 
CDMA systems was thought to be difficult, it has turned out that replica 
method, which is an analytical tool developed in the research field of statis- 
tical physics of disordered systems (spin glasses), is very useful for the anal- 
ysis [sl, More precisely, these studies have revealed that replica method 
allows us to evaluate "macroscopic" performance of CDMA systems with 
independent and identically-distributed (i.i.d.) random spreading sequences 
in the large-system limit, such as mutual information between inputs and 
outputs, bit error rate, and so on. 

Since theoretical performance of CDMA systems is affected by choices 
of spreading sequences, design of spreading sequences is an important prob- 
lem in CDMA. There have been several researches in which the problem 
of designing spreading sequences is formulated in terms of an optimization 
problem. For example, Rupf and Massey [1] discussed optimization of spread- 
ing sequences with the channel capacity of CDMA systems as a measure of 
optimization. They showed that so-called Welch bound equality (WBE) 
spreading sequences, which minimize the total squared correlation (TSC) of 
spreading sequences, achieve the channel capacity. 
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We restrict ourselves to considering the optimization problem of spreading 
sequences of CDMA systems with the mutual information between inputs and 
outputs as the measure of optimization. There have been several researches in 
which such optimization problems are discussed [s], 0, 0] ■ They have assumed 
inputs of the system to be Gaussian distributed and discussed theoretical up- 
per bound (i.e., channel capacity) of mutual information between inputs and 
outputs. If one wishes to consider realistic wireless communication systems, 
however, it is important to study the optimization problem under the as- 
sumption of non-Gaussian inputs. The objective of this paper is therefore to 
discuss the optimization problem of spreading sequences of CDMA systems 
when one allows non-Gaussian inputs. We would like to emphasize that, un- 
like previous statistical-mechanics studies of CDMA systems js], 0, [sj whose 
objectives are basically to analyze macroscopic system performance, we show 
in this paper that the statistical-mechanics approach is also useful in dealing 
with design problems in wireless communication, with the optimization prob- 
lem of spreading sequences of CDMA systems as a demonstrative example. 
A digest version of this paper has been presented as a conference paper [9j. 

2. Problem 

We consider the following real- valued i^-user CDMA channel model, 



where is an information symbol of user k. We assume that {xk, k = 
1, ■ ■ ■ , K} are i.i.d. random variables, and let p{-) be the prior probability of 
Xk, whose mean and variance are assumed to be zero and one, respectively. 
{s^k, = 1, ■ ■ ■ 5 is the spreading sequence of user k in the symbol interval 
of interest, and L denotes the spreading factor of the CDMA channel model. 
We assume that the power of the spreading sequences is normalized to one, 
so that J2u= 

^{s^k/VLf = 1 holds for A; = 1, ■ ■ ■ , ir. We assume additive 
white Gaussian noise (AWGN): ~ -^(0, 1) so that cr^ is the variance of 
AWGN. Let us introduce the following notations: y = [yi,--- ,1/1]'^, n ^ 
[ni, ■ ■ ■ , ul]'^, X = [xi, ■ ■ ■ , xk]'^, and 5* = (S^k), S^k = {l/y/L)s^k, k = 
1, ■ ■ ■ , K] fj, = 1, ■ ■ ■ , L. The system model ([1]) is then rewritten as 




1 



(1) 



y = Sx + an. 



(2) 
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One can consider a maximization problem of per user mutual information 
between x and y with respect to the spreading sequences S", with the channel 
input X drawn from the probability distribution p{x) = Yli.p{xk), 



user 



Hx;y)\s, 



(3) 



where the notation I{x; y)\s denotes the mutual information between x and 
y when S is specified. When K < L, the mutual information Cuser is niaxi- 
mized by assigning to all users orthogonal L-dimensional vectors as spreading 
sequences, regardless of the input distribution p(-). When > L, on the 
other hand, spreading sequences maximizing the mutual information Cuser 
are not trivial. When Xk, k = 1, ■ ■ ■ , K, are i.i.d. standard Gaussian ran- 
dom variables, it is known that the WBE spreading sequences maximize the 
mutual iuformafou C„.„ & WBE spreaiiug sequences are characterized 



where Ilxl is an L-dimensional identity matrix. 

When one assumes Gaussian inputs, spreading sequences maximizing 
Cuser have been identified in more general system models than ([2]). For exam- 
ple, in a system model where one allows the power of inputs to be different, 
Viswanath and Anantharam showed that the mutual information is max- 
imized by assigning orthogonal spreading sequences to relatively high-power 
users and so-called generalized WBE spreading sequences to the remaining 
users, where the users are classified according to a certain criterion. Also, in 
a system model where the inputs may arrive asynchronously, Luo et al. [3] 
showed that the mutual information is maximized by spreading sequences 
which can be regarded as an extension of the ones which Viswanath and 
Anantharam proposed. 

On the other hand, when x^'s are drawn from a non-Gaussian distribu- 
tion, to the authors' knowledge, spreading sequences maximizing the mutual 
information ([2]) have not been known. We analyze, via statistical mechanics, 
spreading sequences maximizing the mutual information of the system with 
non-Gaussian inputs. Since the case with non-Gaussian inputs is difficult 
to analyze analytically, we resort to making several assumptions. First, we 
evaluate mutual information in the large-system limit, in which the number 
of users K and the spreading factor L are both sent to infinity while main- 
taining their ratio (3 = K/L constant. Second, we assume random spreading. 
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More specifically, we assume that the sample correlation matrix R = S'^S 
of random spreading sequences S is asymptotically orthogonally invariant, 
that is, the probability law of R and that of an orthogonal transform U^RU 
are the same for any orthogonal matrix U in the large-system limit, and 
that empirical eigenvalue distribution of R converges to a limiting eigenvalue 
distribution p(A) with a finite support included in [Amin, Amax], in the large- 
system limit. Under the assumption of random spreading, we consider the 
average conditional mutual information in the large-system limit, 

C= lim E5{ascr}= lim ^I{x;y\S), (5) 

where E5 denotes expectation with respect to S", and where I{x]y\S) is 
conditional mutual information between x and y given 5*. We discuss maxi- 
mization of C with respect to characteristics of the random matrix S. 



3. Analysis 

3.1. Evaluation of average mutual information via replica method 
The average mutual information ([5]) is decomposed into two terms, 

C = hm 1 [¥.y^s{\ogp{y\S)} - Ey,^^s{^ogp{y\x, S)}] (6) 

= -^-^(l + log(2vra2)), (7) 

with 

hm ^Ey,s{hgpiy\S)}, (8) 

where Ey^s denotes expectation with respect to y and 5*. Direct calculation of 
the right-hand side of ([8]) is in general computationally intractable. In order 
to evaluate ([8]), we invoke the replica method. Substituting the identity 

\imJ-{p{y\S)r = \ogp{y\S) (9) 

ra— »0 On 

to the right-hand side of ([8]), we obtain 

^ = - lim 1 limJ-\ogEy,s{{p{y\S)r}. (10) 

K^oo K n^O On 
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We assume that the hmit with respect to K and the hmit and the differenti- 
ation with respect to n are interchangeable without altering the final result, 
obtaining 

^ = - hm |- hm 1 logEy^s{{p{y\S)r}. (H) 

n^O on K^oo A 

The limit K ^ oo allows us to apply the saddle-point method to evaluate a 
relevant quantity. We apply replica trick in order to evaluate ffTTl) . in which 
we first evaluate ¥.y^s{{p{y\S)Y} assuming that n is a non-negative integer, 
and then perform the limit and the differentiation with respect to n, assuming 
that n is real. 

Evaluation of (fTTi) basically goes in a similar manner as [8|. Detailed 
analysis is described in the appendix. Here, we only show the result. The 
average mutual information in the large-system limit is given by 

C = -\e8 - \g (^-^ _ i log 1^ - 1 - I p{u- 6) \ogp{u- 6) du, (12) 

where {£, 9} are parameters whose values are to be determined later, and 
where p{u] 9) is a probability density function of output u of a scalar AWGN 
channel with 1/9 the noise variance, when the channel input x is generated 
from the distribution p{x). The function G{t) is defined as 

G{t) = [ R{z) dz (13) 
Jo 



where H{z) is the R-transform [12] of the limiting eigenvalue distribution 
p(A) of the correlation matrix R, which is defined on the basis of the Hilbert 
transforn(§ of p(A), 

C(7) = I :^dA, 7< Amin, (14) 

as 

c (niz) + ^)=z. (15) 



^It should be noted that the so-called Cauchy transform is defined by the same formula 
as the Hilbert transform (|14p . but with 7 in the upper half of complex plane. 
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The parameters {S, 6} are to be determined from the following saddle-point 
equations: 



£ = E{{x- {x) f; 9} 



(16) 
(17) 



where (x) denotes posterior mean estimate of the channel input x of the 
scalar AWGN channel introduced above, defined as 



{x) 



(18) 



and where E in (1161) denotes expectation with respect to the channel input 
X and output u of the scalar AWGN channel. 





A A 

Figure 1: Eigenvalue distributions p'^^(left figure) and p^^^(right figure) with (3 = 1.5. 
Botli distributions have probability weight 1 — l//3atA = 0. 

The average mutual information (fT2!) depends on the limiting eigenvalue 
distribution p(A) of R as well as the prior distribution p{xk)- When we 
assume that s^k are i.i.d. random variables whose mean and variance are 
zero and one, respectively, our result is reduced to that obtained by Guo and 
Verdii j^. In this case, the limitin g ei genvalue distribution p(A) is given by 



the so-called Marcenko-Pastur law 13 



P^^(A) 



(19) 
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where (x)"*" = max(0,x), and a = (1 — 6 = (1 + (see figured]), 

whose R-transform is given by 

- (20) 

Substituting (l20|l to ( fT3|l . one can confirm the above-mentioned fact. Our 
analysis also includes the case of the system with WBE spreading sequences. 
Since the characteristic of WBE spreading sequences is expressed as (jll), the 
correlation matrix of WBE spreading sequences has trivial zero eigenvalue 
and the eigenvalue A = /?, with multiplicities {K — L) and L, respectively 
(see figured]), and therefore 

p'^''''(A) = (l-^)5(A) + ^5(A-/5), (21) 

whose R-transform is given by 

R™(^) = (22) 

One can evaluate, via TV^^^^z), the mutual information when orthogonally- 
invariant random WBE spreading sequences are employed. In figure [21 we 
show a comparison of the mutual information when the above two spreading 
sequences are employed, and when probability distribution of {xk} is given by 
p{xk) = {6{xk — 1) + S{xk + l))/2, k = 1, ■ ■ ■ ,K. One can confirm that WBE 
spreading sequences achieve higher mutual information than i.i.d. random 
spreading sequences do. 

3.2. Optimizing spreading sequences 

Choices of spreading sequences affect the average mutual information C 
through the limiting eigenvalue distribution p(A) of the correlation matrix 
R. Then, we regard the average mutual information C as a functional of 
p(A), and seek the eigenvalue distribution p*(A) which maximizes the average 
mutual information C. Hereafter, we consider the case of /3 > 1 since optimal 
spreading sequences in the case of /? < 1 are obviously orthogonal spreading 
sequences. In optimizing C with respect to p(A), the following two constraints 
should be imposed on p(A): First, since (3 > 1, the KxK matrix R has trivial 
zero eigenvalues with multiplicity {K — L) . Second, since we have normalized 
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SNR [dB] 

Figure 2: The average mutual information C when p^BE (\yBE) and (Random) are 
specified, in case of /3 = 1.5. 



the power of spreading sequences as = 1, = 1, ■ ■ ■ , i^', the 

matrix S should satisfy 



K 



TiS'^S = ^\k = K, (23) 



fc=i 



where {A^} are the eigenvalues of K x K matrix S'^S. In terms of p(A), the 
constraint (123!) is expressed as 

j Ap(A)dA = 1. (24) 

We rewrite p(A) in view of these constraints as 

P(A)= (l-^)5(A) + ^7r(A), (25) 

where 7r(A) satisfies 

y"7r(A)dA = l, (26) 
as the normalization as a probability distribution, and 

A7r(A)dA = /3, (27) 



which corresponds to the normahzation of the power of spreading sequences fl24l) . 

In order to discuss the extremum of C with respect to p(A), we consider 
first-order perturbations of C. Since the parameters affected by the pertur- 
bation of p(A) are {G{t), £, 6*}, the functional derivative of C with respect 
to p is expressed as 



Since 6} should satisfy the saddle-point equations ( |T6|) and (fT7|) . the 
derivatives of C with respect to the parameters 6} should be zero at the 
saddle point. Therefore, one can safely ignore the effects of perturbations via 
£ and 6. 

Our next observation is that, if one can find an eigenvalue distribution 
which maximizes —{l/2)G{—£/a'^), which is the only term having the first- 
order effect in fll2l) . it also maximizes the average mutual information fll2p . 
We rewrite —G{—£/a'^) as 



Since —£/a'^ < and R(-z) > 0, one can make the following statement: If 
there is a distribution p*(A) whose R-transform R*(-z) satisfies 



for any R-transform R(-2) of the distribution p(A) which satisfies the con- 
straints fl25l) - fl27|) . p*(A) also maximizes —G{—£/a'^). We summarize the 
above arguments in the next proposition. 

Proposition 1 If one can find an eigenvalue distribution p*(A) which max- 
imizes R-transform for ^ z G (— £^/o"^,0), p*(A) also maximizes the average 
mutual information G . 

It should be noted that the existence of p*(A) is not guaranteed at this stage. 
However, in the following, we show that there is a distribution which satisfies 
the condition of Proposition 1. 

As a next step, we convert the optimization problem in terms of R- 
transform into the one in terms of Hilbert transform. Since Hilbert transform 
C(7) is a monotonically decreasing function of 7, and since Hilbert transform 



5G _ 5G 5G dG 6£ dG 66 

J^~Jg'J^^^'J^^^'Tp' 



(28) 




(29) 



YC{z) > R{z), for e i-£/a^, 0) 



(30) 
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has the relation (I15p with R-transform, it follows that R(-2) + also de- 
creases monotonically with respect to z. This fact leads us to the following 
statement: If one can find an eigenvalue distribution p*(A) which maximizes 
C(7) for ^7 < Amin, and which satisfies fl25|) -fl271). p*(A) also maximizes R(2;) 
for ^ z G (-Zmin, 0), with 

2;min = lim C(7). (31) 
Summarizing the arguments so far, we can state the next proposition. 

Proposition 2 //one can find an eigenvalue distribution p*(A) whose Hilbert 
transform C* (7) satisfies 

C*(7)>C(7), ^<A^in, (32) 

for any eigenvalue distribution p(A) with Hilbert transform C{'~f), which sat- 
isfies fl25l) - fl27|) ■ p*(A) also maximizes the average mutual information C. 

Following the above proposition, we consider the maximization problem 
of the Hilbert transform for 7 < Amin- Substituting (!25|) to (IT^ . the Hilbert 
transform C{'y) is rewritten as 

Since the first term, which is derived from the trivial zero eigenvalues, has 
no room for optimization, we maximize the second term under the con- 
straints ( l26i) and ( 1271) . Let us consider the following integral: 

^(^) (t^a " ^^^0 ^^^^ 

where /(A) is a linear function tangential to 1/(7 — A) at A = /3. Since the 
function /(A) is a linear function, the expectation of /(A) with respect to 7r(A) 
is constant under the constraints fl26l) and fl27j) . Thus, the quantities flMl) 
and ( !33l) are maximized by the same eigenvalue distribution. We here con- 
sider a maximization problem of the objective function ( IMl) for 7 < Amin 
with the constraint ( l26l) . but without the constraint ( 1271) . Since we have only 
to consider A G [Amin, Amax], we can assume A > 7. Since 1/(7 — A) is convex 
upward in A for A > 7, one has 

^ /(A)<0, A>7, (35) 



7 — A 
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where the equahty holds if and only if A = /3. Then, the objective func- 
tion (IMI) is maximized for 7 < Amin by the probability distribution 

7c{X) =6{X- (3). (36) 

Since the distribution fl36p incidentally satisfies the power constraint fl27p . 
the distribution (1361) is also a maximizer of the objective function (1341) with 
both of the constraints fl26l) and fl27|) . Since the two functions fl33|) and flMl) 
are maximized by the same probability distribution, the distribution fl36|) is 
also the optimal solution of the maximization problem of the Hilbert trans- 
form. Thus, we obtain the maximizer of the average mutual information f|T2|) . 
Substituting fl5B]) to fl2S]) . one can confirm that the optimal eigenvalue distri- 
bution is the one of WBE spreading sequences p^^^, which is given by fl?I]) . 

We have so far shown that WBE spreading sequences are also asymptot- 
ically optimal in CDMA systems with a non-Gaussian input distribution in 
the large-system limit. This finding is an extension of the optimality result 
of WBE spreading sequences for Gaussian-input CDMA systems. 

4. Conclusion 

We have demonstrated that the statistical-mechanics approach is useful 
not only in analyzing theoretical performance of wireless communication sys- 
tems but also in providing clues to how to design them, via the problem of 
optimizing spreading sequences in CDMA systems. We have evaluated, via 
replica method, average mutual information between input and output of the 
system in the large-system limit, and discussed the optimization problem of 
the average mutual information in terms of characteristics of random spread- 
ing sequences. It has been shown that the average mutual information is 
maximized in the large-system limit by orthogonally-invariant random WBE 
spreading sequences even when the inputs are non-Gaussian. Although in 
this paper we have only studied a fully- synchronous CDMA model with per- 
fect power control, one can consider the same problem in more general CDMA 
systems, such as the one with unequal-power users, and that is deferred to 
our future work. 

A. Details of replica analysis 

In this appendix, we explain how to evaluate given by f|TT]) . First, we 
calculate the expectation 'Kys{p{y\S)"'} assuming that is a non-negative 
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integer. Introducing replicated random vectors Xa = [xai,-'' i^axY ^ 
M'^, a = 0, ■ ■ ■ , n, which are drawn from the same probabihty distribution 
as X, we rewrite Ey^s{p(y|5')"'} as 



IE5 jy j \\p{y\Xa, S)p{Xa)<\Xa<ly 



(37) 



Performing the integral with respect to we obtain 

^yA{p{y\s)T] 

( \K L riL 

= E,„, J exp -TiRV - - log(n + 1) - ^ log(27ra^ 

where K x K matrix V is given by 
1 



(38) 



V 



{n + l)Ka'^ 



II- \ / \ 1 



,a=0 



.a=0 



(39) 



a=0 



The expectation with respect to 5" can be performed via the so-called Itzykson- 
Zuber integral 14, l3] (see also [sl, [ll]), since R = S'^S is assumed orthogo- 
nally invariant and rank of V is at most (n + 1), as 



lim — logEs <; exp 

K — »oo J\ 



K 



TiRV 



-TrG(y), 



(40) 



where G{x) is the function defined in (|T3l) . Thus, we obtain the following 
equation, ignoring vanishing terms in the large-system limit. 



^y,smy\s)T} 

= E{^4 I exp 

We next take expectation of fj4T|) with respect to {Xa}. Since eigenvalues 
of the matrix V are functions of {Xa} only through their inner products 
Xa - Xh, a,b = 0, ■ ■ ■ ,n, we rewrite the expectation with respect to {Xa} into 
the one with respect to the (n + 1) x (n + 1) matrix 

1 ^ 

Q = {Qab), Qab = -^^XakXbk, (42) 

k=l 



K T nT 

-TiGiV) - - \og{n + log(27ra2) 



(41) 
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as 



exp[ir6;(g)]//^(g)dg, 

where KQ{Q) is the exponent of fl^Tl) . 



Q{Q) 



-TrG(\/) 



1 T) 

(Q)-^log(n+l)--log(27ra2) 



(43) 



(44) 



and where [ik is the following measure 



K 



MQ) = i n M E 

KQab 



(45) 



.0<a<fe \fc=l 



Utilizing the saddle-point method [3l,0|, we evaluate (H3!) in the limit iiT — > oo 



as 



hm llogE{(p(y|5))"} = sup{6;(Q) - J(Q)}, 



(46) 



where T{Q) is the rate function of the empirical means (142|) . defined via a 
Legendre transform as 



I{Q) = sup 



J2 Q abQab- log M{Q) 



.0<a<b 



(47) 



where Q = (Qab) is an (n + 1) x (n + 1) symmetric matrix. The cumulant 
generating function logM(Q) of {xa} is defined as 



logM(Q) = log%,4<;exp 



^ ^ Qab-^a 



Xb 



.0<a<b 



(48) 



In order to proceed further, we assume the so-called replica symmetry: 
We assume that the extremum of fH^ is invariant under exchanges of the 
replica indexes. Under the assumption of replica symmetry, we introduce 
new parameters, 

Qaa = P, Qab = Q, a ^ b. (49) 
Using these parameters, the eigenvalues of V are expressed as 



Ai 



p-q 



(50) 
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A2 = 0, (51) 

whose multiplicities are n and {K — n), respectively. Since G{x) is an analytic 
function and G{0) = 0, we can express G{Q) as 



n 



G{Q) = if 



p-q 



-— log(n+l)-— log(27r(T^) 



We also apply the assumption of replica symmetry to Q as 

Qaa = C, Qab = CI ^ b, 

and rewrite the rate function 2{Q) as 

n{n + 1) 



(52) 



(53) 



I{Q) = sup 



[n + l)cp + 



-9q 



logE{x4< exp 



Xb 



a=0 



0<a<b 



(54) 



Deriving the extremum condition of fl46l) with respect to p and q in the limit 
n 0, we obtain the following saddle-point equations, 

c = 0, 



e 



1 



R( 



a2 ' 



(55) 
(56) 



Similarly, as the extremum condition of fl46|) . we obtain the following saddle- 
point equation, 

p-q = ^{{x~{x)f-e}, (57) 

where x follows the probability distribution p(x), and where (x) is the poste- 
rior mean estimate of channel input x given output u in a single-user AWGN 
channel whose variance is 1/9 and whose input and output are x and u, 
respectively. Introducing the posterior variance S = p — q, we obtain the 
saddle-point equations (fT6!) and (fT7|) . In order to obtain JF, we differentiate 
logE{(p(t/|S'))"}, which is obtained by substituting (15^ and flMj) to 
with respect to n, and then take the limit n ^ 0. Finally, we obtain the 
representation of JF as 



1, 27r 



^- I p{u- 9) \ogp{u- 9)du + ^{l + log(27ra2)). (58) 
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Substituting (155]) to ([7]), we obtain the average mutual information (fT^ . 
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